{
 "cells": [
  {
   "cell_type": "code",
   "id": "initial_id",
   "metadata": {
    "collapsed": true,
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.614835Z",
     "start_time": "2025-03-03T10:08:19.397884Z"
    }
   },
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ],
   "outputs": [],
   "execution_count": 2
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.620451Z",
     "start_time": "2025-03-03T10:08:19.615839Z"
    }
   },
   "cell_type": "code",
   "source": [
    "df_data = pd.DataFrame([np.random.randn(3),[1.,2.,np.nan],[np.nan,4.,np.nan],[1.,2.,3.]])\n",
    "print(df_data)"
   ],
   "id": "21e014f3745cfffc",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          0         1        2\n",
      "0 -0.016174 -0.128714 -2.48349\n",
      "1  1.000000  2.000000      NaN\n",
      "2       NaN  4.000000      NaN\n",
      "3  1.000000  2.000000  3.00000\n"
     ]
    }
   ],
   "execution_count": 3
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.624960Z",
     "start_time": "2025-03-03T10:08:19.621155Z"
    }
   },
   "cell_type": "code",
   "source": "df_data.iloc[2,0]",
   "id": "18ea9e38fbaeb302",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.float64(nan)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 4
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.630433Z",
     "start_time": "2025-03-03T10:08:19.626379Z"
    }
   },
   "cell_type": "code",
   "source": [
    "\"\"\"\n",
    ".isnull() 是 Pandas 库中一个非常实用的方法，主要用于检测数据中的缺失值（通常表示为 NaN，即 Not a Number），它可以应用于 Pandas 的 Series 和 DataFrame 对象。\n",
    "\"\"\"\n",
    "print(df_data.isnull())  # True 表示缺失值，False 表示非缺失值\n",
    "\"\"\"\n",
    "df_data.isnull().sum() 会对 df_data.isnull() 返回的布尔型 DataFrame 按列进行求和。在 Python 中，布尔值 True 会被当作 1，False 会被当作 0，因此求和的结果就是每列中缺失值的数量。\n",
    "df_data.shape 返回一个元组，其中第一个元素表示 DataFrame 的行数，第二个元素表示列数。所以 df_data.shape[0] 就是 df_data 的行数，也就是数据的总数。\n",
    "用每列的缺失值数量除以数据的总数（即行数），就得到了每列的缺失率。\n",
    "\"\"\"\n",
    "print(df_data.isnull().sum()/df_data.shape[0])"
   ],
   "id": "9ffaf18e3126b01c",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       0      1      2\n",
      "0  False  False  False\n",
      "1  False  False   True\n",
      "2   True  False   True\n",
      "3  False  False  False\n",
      "0    0.25\n",
      "1    0.00\n",
      "2    0.50\n",
      "dtype: float64\n"
     ]
    }
   ],
   "execution_count": 5
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 删除缺失数据",
   "id": "188ff5571b6b3426"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.636507Z",
     "start_time": "2025-03-03T10:08:19.631219Z"
    }
   },
   "cell_type": "code",
   "source": [
    "\"\"\"\n",
    ".dropna() 是 Pandas 库中另一个实用的方法，用于删除 DataFrame 中包含缺失值的行。\n",
    "语法：\n",
    "对于 DataFrame：DataFrame.dropna(axis=0, how='any', thresh=None, subset=None, inplace=False)\n",
    "对于 Series：Series.dropna(axis=0, inplace=False)\n",
    "参数解释：\n",
    "axis\n",
    "取值为 0 或 'index'（默认），表示按行处理，即移除包含缺失值的行。\n",
    "取值为 1 或 'columns'，表示按列处理，即移除包含缺失值的列。\n",
    "how\n",
    "取值为 'any'（默认），表示只要行或列中存在一个缺失值，就将该行或列移除。\n",
    "取值为 'all'，表示只有当行或列中的所有值都为缺失值时，才将该行或列移除。\n",
    "thresh\n",
    "整数类型参数，用于指定非缺失值的最小数量。如果行或列中的非缺失值数量小于该阈值，则将该行或列移除。\n",
    "subset\n",
    "列表类型参数，指定在哪些列或行（取决于 axis 参数）中检查缺失值。仅对指定的列或行进行缺失值检查和移除操作。\n",
    "inplace\n",
    "布尔类型参数，默认为 False。如果为 True，则直接在原对象上进行修改，不返回新对象；如果为 False，则返回一个移除缺失值后的新对象。\n",
    "\"\"\"\n",
    "print(df_data)\n",
    "print('-'*50)\n",
    "print(df_data.dropna())  # 删除缺失值所在的行\n",
    "print('-'*50)\n",
    "print(df_data.dropna(subset=[0]))  # 代码会查看列索引为 0 的这一列，将该列中包含缺失值的行从 df_data 中移除。"
   ],
   "id": "ccc34ef7fed89fe4",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          0         1        2\n",
      "0 -0.016174 -0.128714 -2.48349\n",
      "1  1.000000  2.000000      NaN\n",
      "2       NaN  4.000000      NaN\n",
      "3  1.000000  2.000000  3.00000\n",
      "--------------------------------------------------\n",
      "          0         1        2\n",
      "0 -0.016174 -0.128714 -2.48349\n",
      "3  1.000000  2.000000  3.00000\n",
      "--------------------------------------------------\n",
      "          0         1        2\n",
      "0 -0.016174 -0.128714 -2.48349\n",
      "1  1.000000  2.000000      NaN\n",
      "3  1.000000  2.000000  3.00000\n"
     ]
    }
   ],
   "execution_count": 6
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.640269Z",
     "start_time": "2025-03-03T10:08:19.637300Z"
    }
   },
   "cell_type": "code",
   "source": "print(df_data.dropna(axis=1))  # 删除缺失值所在的列",
   "id": "83f8ec3a9917c1b2",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          1\n",
      "0 -0.128714\n",
      "1  2.000000\n",
      "2  4.000000\n",
      "3  2.000000\n"
     ]
    }
   ],
   "execution_count": 7
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### 填充缺失数据",
   "id": "4cf5780795748e0"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.644776Z",
     "start_time": "2025-03-03T10:08:19.641086Z"
    }
   },
   "cell_type": "code",
   "source": [
    "print(df_data.iloc[:,0].fillna(-100.))\n",
    "print('-'*50)\n",
    "print(df_data)"
   ],
   "id": "41d365fbbb45b0dd",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0     -0.016174\n",
      "1      1.000000\n",
      "2   -100.000000\n",
      "3      1.000000\n",
      "Name: 0, dtype: float64\n",
      "--------------------------------------------------\n",
      "          0         1        2\n",
      "0 -0.016174 -0.128714 -2.48349\n",
      "1  1.000000  2.000000      NaN\n",
      "2       NaN  4.000000      NaN\n",
      "3  1.000000  2.000000  3.00000\n"
     ]
    }
   ],
   "execution_count": 8
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.648443Z",
     "start_time": "2025-03-03T10:08:19.645551Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 依次拿到每一列\n",
    "print(df_data.columns)\n",
    "print('-'*50)\n",
    "for i in df_data.columns:\n",
    "    print(i)\n",
    "    print(df_data.loc[:,i])\n",
    "    print('-'*50)"
   ],
   "id": "5c1da281ab345e27",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RangeIndex(start=0, stop=3, step=1)\n",
      "--------------------------------------------------\n",
      "0\n",
      "0   -0.016174\n",
      "1    1.000000\n",
      "2         NaN\n",
      "3    1.000000\n",
      "Name: 0, dtype: float64\n",
      "--------------------------------------------------\n",
      "1\n",
      "0   -0.128714\n",
      "1    2.000000\n",
      "2    4.000000\n",
      "3    2.000000\n",
      "Name: 1, dtype: float64\n",
      "--------------------------------------------------\n",
      "2\n",
      "0   -2.48349\n",
      "1        NaN\n",
      "2        NaN\n",
      "3    3.00000\n",
      "Name: 2, dtype: float64\n",
      "--------------------------------------------------\n"
     ]
    }
   ],
   "execution_count": 9
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-03T10:08:19.653108Z",
     "start_time": "2025-03-03T10:08:19.649486Z"
    }
   },
   "cell_type": "code",
   "source": [
    "print(df_data)\n",
    "print('-'*50)\n",
    "print(df_data.dropna(axis=1,inplace=True))\n",
    "print('-'*50)\n",
    "print(df_data)"
   ],
   "id": "a7961d13d7e0cc09",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          0         1        2\n",
      "0 -0.016174 -0.128714 -2.48349\n",
      "1  1.000000  2.000000      NaN\n",
      "2       NaN  4.000000      NaN\n",
      "3  1.000000  2.000000  3.00000\n",
      "--------------------------------------------------\n",
      "None\n",
      "--------------------------------------------------\n",
      "          1\n",
      "0 -0.128714\n",
      "1  2.000000\n",
      "2  4.000000\n",
      "3  2.000000\n"
     ]
    }
   ],
   "execution_count": 10
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
